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ABSTRACT 


Given  data  z , 


gtt^)  +  e^,  1  <  i  <  n,  where  g  is  the  unknown 


function,  the  t^  are  unknown  d-dimensional  variables  in  a  domain  12,  and  the 

e  are  i.i.d.  random  errors,  the  smoothing  spline  estimate  g  .  is  defined 
i  -12 

to  be  the  minimizer  over  h  of  n  £(z. -h(t. ))  +  Xj  (h) ,  where  X  >  0  is  a 

i  i  m 

smoothing  parameter  and  Jm(h)  is  the  sum  of  the  integrals  over  12  of  the 
squares  of  all  the  111th  order  derivatives  of  h.  Under  the  assumptions  that 
12  is  bounded  and  has  a  smooth  boundary,  X  ♦  0  appropriately,  and  the  t^ 
become  dense  in  (2  as  n  ♦  •,  bounds  on  the  rate  of  convergerence  of  the 
expected  square  of  order  Sobolev  norm  (L2  norm  of  pth  derivatives), 

are  obtained.  These  extend  known  results  in  the  one  dimensional  case.  The 
method  of  proof  utilizes  an  approximation  to  the  smoothing  spline  based  on  a 
Green's  function  for  a  linear  elliptic  boundary  value  problem.  Using 
eigenvalue  approximation  techniques,  these  rate  of  convergence  results  are 


extended  to  faily  arbitrary  domains  including  (2 
case  p  *  0,  i.e.  L2  norm. 


R  ,  but  only  for  the 
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SIGNIFICANCE  AND  EXPLANATION 


Smoothing  spline*  are  used  to  approximate  smooth  functions  when  there  are 
only  available  noisy  values  of  the  function  at  discrete  values  of  the 
independent  variables*  It  is  shorn  herein  that  as  the  grid  of  values  of  the 
independent  variables  becomes  denser  in  the  region  of  interest,  the  smoothing 
spline  estimate  approaches  the  true  function.  Results  on  the  rate  of  this 
convergence  are  given.  Convergence  of  derivatives  is  investigated,  also,  but 
under  the  assumption  that  the  region  is  bounded.  The  theory  of  linear 
elliptic  partial  differential  equations  is  used  extensively,  along  with 
eigenvalue  approximation  methods. 
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CONVERGENCE  RATES  TOR  MULTIVARIATE 
SMOOTHING  SPLINE  FUNCTIONS 

Dennis  D.  Cox* 

1.  Introduction. 

Let  Q  be  a  domain  In  d-dimensional  Euclidean  apaca  Rd.  Suppose  that  there  is  an 
unknown  real  valued  function  g  defined  on  0  for  which  the  following  data  is  available 

(t.U  *k  "  9<V  +  *k  k  “  , 

where  t. ,t., . . . ,t  are  known  points  in  0  (referred  to  as  "knots"),  and  e,,e  ,,...,e 

1  2  II  x  c  n 

are  (unknown)  random  errors.  We  would  like  to  obtain  an  estimate  of  g  from  the  data 
vector  jt  e  *n.  If  it  is  known  a  priori  that  g  is  a  smooth  function,  then  a  smoothing 
spline  will  provide  a  reasonable  estimate  of  g.  These  have  been  widely  used  for  such 
estimation  problesm  (see  e,g,  Wahba  [21]  or  Ragosin,  at.  al.  (101),  and  can  be  justified 
statistically  as  either  optimal  Bayas  estimates  (Ximaldorf  and  wahba  [7))  or  pointwise 
minimax  estimates  (Speckman  [13]),  In  order  to  describe  these  estimates,  some  notation  is 
needed. 

A  multi-index  a  €  *d  is  a  d-vector  whose  coordinates  belong  to  S+  “  jo, 7,2,,.,}, 
The  order  of  a,  denoted  |a|,  is  given  by 


Iml 


Given  a  e  Zd  with  |a|  -  m,  there  is  an  associated  partial  differentiation  operator  of 
order  m  given  by 


D 


:  ii 
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For  any  domain  Q  C  Rd,  lat  W^(fl)  for  ■  «  0,1,...  denote  the  Sobolev  space  of  all 

generalized  functions  (Schwarz's  distributions)  for  which  all  partial  derivatives  of 

order  <  m  exist  in  L^tQ).  (For  a  more  formal  definition,  consult  Adana  [1],  Agaon  [2], 

or  Triebel  [17].)  Letting  <• , •>Q  denote  the  usual  inner  product  on  L2<Q)  ■  W°(Q),  we 

define  the  inner  product  of  W^(fl)  by 

<f,g>  -  l  <Daf,D°g>  , 

“  |a|<m  0 

where  the  sun  is  over  all  aulti-indices  a  of  order  «  a.  The  associated  norm  is 

HI,  -  (<f,f>a)1/2. 

With  these  definitions,  W^(fl)  is  a  Hilbert  space. 

A  related  seal-inner  product  is 

(f.g)  -  I  <Oaf,D°g>0* 

|o|»m 

which  gives  rise  to  the  aeainora 

if|a  * 

A  norm  equivalvant  to  1*1^  which  is  often  convenient  is 

(1.2)  1 11*1  II-  -  (l«l»  ♦  I  iD^flV2, 

k»1 

where  is  the  operator  for  partial  differentiation  in  the  k^"  variable.  See  Theorem 
4.2.4  of  Triebel  [17].  He  shall  frequently  need  other  norms,  in  which  case  the  space  on 
which  the  nor*  is  defined  and  finite  will  appear  as  a  subscript  to  the  norm  symbol. 

The  smoothing  spline  estimate  of  g  can  now  be  described.  First  make  the  following 

Assumption  1.  The  data  vector  z  is  given  by  (1.1)  where  ei » e2 » • • • *  en  are  uncorrelated 

2 

random  variables  with  aean  0  and  positive,  finite  variance  a  . 

Definition  1.1.  Let  a  >  d/2  be  the  order,  &  -{t.,t,,...,t  J  C  0  be  a  set  of  knots,  and 

n  l  2  h  “ 

1  >  0  be  a  saoothing  parameter.  Then  the  smoothing  spline  operator  S  *"■*•  w"(8)  (if 

n  A  2 

it  exists)  is  given  for  {  (  in  by 


-2- 


Sn^£  “  gnX  if  and  only  if  compared 
to  any  h  e  t^(Q)  ,  minimises 

L»X<h>  -  n'1  j  («k  -  h(tk»2  ♦  X|h|2  . 

IP*  I 

R—xrka  1.2(1)  Th«  axiatanca  of  a  uniqua  auch  g  .  la  aaaurad  if  tha  knot  aat  4  la 
~~  n  a  n 

ot-unlsolvent,  aa  shown  in  Propoaition  2.1  below.  A  aat  of  pointa  A  C  Rd  ia  called 

n  “ 

d 

w- unisolvent  if  for  any  polynomial  f  on  R  of  degree  <  a  *  1,  the  condition 

fit^  ”  0  for  all  k,  1  <  k  <  n,  inpliea  ♦(t)  -  0  for  all  t. 

(ii)  The  choice  of  the  maoothing  parameter  1  ia  a  problem  of  conaiderable  intereat 

which  we  do  not  discuss  here.  See  Craven  and  Wahba  [5}  for  one  popular  method. 

( lii)  The  choice  of  m  ia  generally  dictated  by  one 'a  prior  knowledge  of  the 

amoothneae  propertiee  of  g,  or  by  the  uae  that  one  will  make  of  the  eatiaate  g  .,  aa  in 

n  A 

Ragosin,  at.  al.  [10].  He  require  a  >  d/2  in  order  that  the  evaluationa  Mt^)  be  well 
defined. 

(iv)  Por  a  diecuaa ion  and  referencea  on  coaputational  aapecta,  conault  Wendelberger 

[24].  • 

Now  our  aain  intereat  here  ia  in  ahowing  that  gnX  convargee  to  g  in  varioua  norma 

aa  n  ♦  •,  provided  that  g  aatiafiea  certain  conditions,  and  that  1  and  A  vary 

n 

with  n  in  an  appropriate  way.  For  d  »  1,  there  have  been  reaulta  preaented  by  numerous 
authora  (Wahba  [21]’,  Craven  and  Wahba  [5],  Utreraa  [18]  [19],  Speckaan  [14],  Ragosin  [9], 
and  Cox  [4]),  but  for  d  >  1  there  have  ao  far  bean  only  conjectures  (Wahba  [22]).  The 
reaulta  presented  herein  easentially  generalise  all  the  previous  results  for  d  •  1 
(Theorem  5.1).  The  aain  result  states  that  as  n  *  •, 

«',nX  -  ,l2  -  X-P/a0[K^m  +  n-’r^j  . 

The  requirements,  briefly,  are  that  X  ♦  0  appropriately  aa  n  ♦  •.  (Assumption  2 
below),  that  An  becomes  uniformly  densa  in  Q  (Assumption  3),  and  that  fl  be  bounded 
with  smooth  boundary  (Aaavaaption  4).  The  number  q  appearing  in  the  bound  is  determined 


from  g  according  to  its  differentiability  and  satisfaction  of  certain  boundary 

conditions.  These  are  spelled  out  in  Remarks  5.6.  If  9  (  W*(Q),  then  we  may  take  q  «  a. 

The  smoothing  splines  which  are  used  most  frequently  are  computed  for  Q  ”  Rd,  to 

which  the  result  described  above  does  not  apply.  However,  we  show  in  Theorem  6.2  that 

if  6  becomes  dense  in  a  subdomain  of  satisfying  the  above  assumptions,  then 
n 

8  [n_1  l  «9nA(tk)  -  9<tk))2]  «  0U  +  n-1!-^2*)  , 

*■1 

provided  g  e  W^(0) .  if  we  put  X  -  n  ,  then  the  upper  bound  becomes 

0 (n”2"^'2* ^ ),  which  is  the  conjectured  optimal  rate  of  convergence  in  (22).  Ne  note 

that  for  d  -  1,  it  is  immaterial  whether  Q  -  R  or  Q  is  some  finite  open  interval 

containing  the  knots,  since  the  two  minimisation  problems  lead  to  the  same  result.  This  is 

why  the  stronger  results  hold  for  the  unidimensional  case. 

In  order  to  present  our  asaisaptiona,  further  notation  and  terminology  is  required, 
d 

Let  F  (t),  t  e  »  denote  the  cumulative  distribution  function  (c.d.f.),  of  the 

A 

probability  awasure  which  assigns  mass  n"1  to  the  point  tv  for  each  t.e  6  ,  i.e. 

*  k  n 


f  (t) 

n 


k*Vt 


Here,  the  sum  in  the  numerator  is  over  all  k,  1  <  k  4  n,  for  which  all  coordinates  of 

tk  are  <  the  corresponding  coordinates  of  t.  Our  basic  assumption  concerning  the 

knots  is  that  the  sequence  converges  uniformly  (at  a  certain  rate)  to  a  c.d.f.  r 

satisfying  certain  conditions.  Our  notation  suggests  that  6  C  4  if  n  <  m,  but  in 

n  —  m 

fact  it  is  only  the  quantity 

d  -  sup  |  F(t)  -  F  (t)  | 
n  t  n 

which  is  important  asymptotically.  Our  main  requirements  for  X,  A  ,  and  0  are  the 

A 

following: 

Assumption  2.  Suppose  there  is  a  sequence  of  intervals  {[X  ,  A  ] }  (X  <  A  for  all  n) 
such  that  X  «  (X  ,  A  )  for  all  n  sufficiently  large,  and  that 


■4- 


11m  d  X"5d/4*  -  lim  A  •  0 
„♦.  »  n  „♦«  n 

Assumption  3 .  Suppose  the  limiting  knot  distribution  F  has  a  density  f  (  C  (fl)  with 
respect  to  d-dimensional  Lebesgue  measure  such  that  for  all  t  €  fl 

0  <  <1  <  f(t)  <  <2  <  • 

for  see*  constants 

Assumption  4.  Suppose  II  is  a  bounded  simply  connected  open  domain  and  its  boundary 
30  is  c",  i.e.  there  exists  a  finite  open  covering  {0^ of  30  and  infinitely 
differentiable  bijections  f£  1  0i  ♦  Rd  for  which  f1(01  n  3S)  C  {*  e  >d  s  xd  -  0}. 

The  following  result  is  an  isms  ill  ate  consequence  of  Theorem  5  •  1  • 

Corollary  1«3«  Suppose  that  Assumptions  1  through  4  hold,  that  m  >  3d/2,  and  that 
g  e  w”(0).  Then  for  any  p(«*  satisfying  p  <  m. 

Big  .  -  ,l2  -  0(X<Bep)/"  ♦  B-’X-<2P+0>/2“,  . 
nA  p 

Furthermore,  if  X  -  Cn-2m/(3m*d)  f{jj.  iooB  congtant  c,  then 

,2  _  Af  -2(m-p)/(2nrt-d)  1 

Bl,nx  -  glp  -  0(n  ) 

Remarks  1.4(1)  We  conjecture  that  the  latter  rate  of  convergence  is  the  best  possible  for 
general  g  e  w£  (0).  Under  a  somewhat  different  model,  stone  [16]  showed  that  for  any 

fixed  t  e  0,  -  [2(m-p)/(2m+d)J  log  n  is  the  optimal  rate  of  convergence  in  probability 

2 

of  log  |d  (t)  -  g(t) |  ,  where  9  (t)  is  any  sequence  of  estimates  of  g(t).  Besides  the 
n  n 

fact  that  our  results  are  global,  whereas  Stone's  are  local.  Stone  uses  a  model  wherein 
(t.,  ,Zj ) ,  • .  • ,  (tn,zn)  are  a  sequence  of  independent  and  identically  distributed  random 
vectors,  and  also  makes  stronger  assumptions  on  the  mth  derivative  than  just 
g  e  w™  (0).  For  d  »  1  and  the  same  observation  model,  Speckman  [15]  has  obtained  the 
result  that  the  best  possible  rate  of  convergence  of 

1  n  2 

log  Bn”'  l  [f<t  )  -  g(t  )f 
k"1  n  *  K 

is  -(2m/(2m+1 )]  log  n,  where  t  is  restricted  to  the  class  of  linear  estimates,  and 


g  e  Wj (a,b).  Arguments  given  below  will  show  that  Speckman's  seminorm  is  asymptotically 


•quivalant  to  I .  Ig .  Specloaan  even  conetructe  an  eatiaate  which  achieves  the  exact 
optimal  rate  of  convergence  (conetante  Included).  Hie  eatiaate  ia  also  a  spline  function, 
but  one  that  is  in  general  saoother  than  the  saoothing  splines  we  deal  with  here. 

(ii)  If  one  strengthens  or  weakena  the  assumption  g  e  Wq(0)  then  faster  or  slower 
rates  of  convergence  are  obtained,  but  soste  new  complications  are  introduced.  The  proper 
function  spaces  to  use  are  Nq(Q)  defined  in  Section  3,  which  are  Besov  spaces  with 

fli 

boundary  conditions  (Section  4.3.3  of  [17]).  For  nonnegative  integers 
q  <  a,  we  have  Nq  ■  w^*,  but  the  spaces  Nq  are  defined  for  all  real  q,  so  one 

must  deal  with  ’fractional"  order  Sobolev  spaces.  If  q  <  ■  +  Vj  then  no  boundary 
conditions  are  active.  However,  for  m  +  V2  <  q  <  3m  +  V2  ,  certain  "natural"  boundary 
conditions  of  the  form  B^g  -  0  are  required  for  g  to  be  in  Nq(8).  These  are 
derived  in  Proposition  2.2.  For  3a  +  V2  <  q,  still  more  boundary  conditions  come  into 

play.  How  if  p  <  2a  -  3d/2,  if  g  e  Nq(0)  where  q  satisfies 

d  <  q  <  2a  +  p,  and  q  >  p,  and  if  X  «  Cn  then 

E.g^-gl2-  0(n-2<q-p>/,2,I*d,>  , 

and  this  is  the  best  obtainable  rate  of  convergence  according  to  our  results.  The 
difficulties  with  boundary  conditions  are  spelled  out  in  greater  detail  in  Remarks  5.6 
below.  From  a  practical  point  of  view,  it  seems  unlikely  that  our  unknown  function  g 
will  satisfy  any  of  the  boundary  conditions.  Hence,  if  the  function  is  very  smooth 
(say  g  e  W^'(Q)),  then  g  €  Nq(fl)  for  all  q  <  a  ♦  V2  ,  end  so  we  can  only  slightly 
improve  the  rate  of  convergence  given  in  Corollary  1.3.  This  difficulty  with  boundary 
conditions  was  first  studied  by  Rice  and  Rosenblatt  [11],  for  d  -  1. 

While  the  results  presented  here  are  significant  in  and  of  themselves,  they  also  have 
applications  to  other  areas.  One  can  easily  see  that  the  results  on  the  generalised  cross 

validation  method  for  choosing  X  as  presented  in  Craven  and  Wahba  [5)  can  be  both 

rigorised  and  generalised.  As  discussed  below,  along  the  say  to  proving  the  main  theorem, 
we  develop  an  approximation  to  which  may  prove  to  be  useful  numerically  when 

smoothing  large  data  sets  (n  >  200,  say).  Silverman  (12)  and  others  have  considered 
penalised  likelihood  estimates  of  probability  density  functions,  and  we  believe  the  methods 


of  this  work  can  bo  applied  to  thia  problem  as  well,  further  atatietical  ramifications  of 
theae  results  will  be  treated  elsewhere. 


We  now  briefly  describe  the  main  ideas  used  in  the  proof  of  Theorem  5.1.  first,  we 
define  a  continuous  analog  of  the  smoothing  spline. 

Definition  1.5.  Let  a  >  d/2  be  the  order,  1  >  0  a  smoothing  parameter,  and  suppose 
F  ia  a  c.d.f.  satisfying  Assumption  3.  Then  the  continuous  smoothing  spline  operator 
i  Lj(0)  ♦  W^(U)  if  given  for  (  e  L2<B)  by  C  “  g^  if  and  only  if  compared  to  any 
other  h  «  W2*<aj,  g^  minimises  1^00  •  /  <?(t)  -  h(t))2df(t)  +  X|h|2  . 

Remark.  The  existence  of  g^  is  shown  in  Proposition  2.2  below. 

The  idea  that  could  be  used  to  approximate  SnX  has  occurred  to  many  authors 

(Cogburn  and  Davis  [3],  Otraras  [18],  Speckaan  [14],  Ragosin  [9]),  but  it  was  in  [4],  where 
a  rather  explicit  perturbation  formula  for  in  terms  of  Sx  was  developed.  As  is 

shown  in  Section  4  below,  there  is  a  function  G^i  Q  x  Q  ♦  R  such  that  for  all  5  e  h^lQ) 
(1.3)  <SA0<t>  -  /^Sx(t,t)5(t)dt 


Now  since  SnX  is  a  linear  operator  (it  ia  obtained  by  minimising  a  quadratic  form), 
may  be  represented  in  the  form 

(Snx5)(t)  -  n'1  J  9BJUt<tlsx 

where 


(1.4) 


*nAk  '  “SnA' 


it 


e^  being  the  kth  coordinate  vector  in  *d.  Noting  that  the  representation  for  SnX 
involves  an  integral  with  respect  to  Fn,  which  approaches  F,  it  is  reasonable  to  expect 
that  as  n  ♦  •,  9nXk<t)  approaches 

(1.5)  V*>  *  Vt'V/fU)  "  0X(Vt,/f(t)  ‘ 

We  will  see  that  even  though  A  is  varying  with  n,  g^  is  in  fact  a  very  good 
approximation  to  gnXk»  provided  X  varies  in  such  a  way  that  consistency  is  obtained.  A 
precise  statement  on  the  accuracy  of  the  approximation  of  gnXk  by  g^  is  given  in 
Theorem  4.3  below. 
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3.  Variational  Problems.  In  this  sac t Ion,  wa  show  tha  axlstanca  and  uniqueness  of  tha 

smoothing  spline  and  continuous  smoothing  splina  as  defined  above.  Even  more  importantly, 

characterisations  of  these  functions  are  given  which  will  be  used  in  the  proof  of  the  main 

theorem.  For  convenience,  define  the  n-vector 

(2.1)  3(A)  -  (g(t,),g(t  ),..., g(t))a 

n  i  z  n 

for  any  continuous  g  j  fl  ♦  R. 

Proposition  2.1.  Suppose  that  A^  is  m-uniaolvent,  and  that  m  >  d/2.  Then  g^  =  S^^z 
exists  for  all  z  e  and  is  the  unique  solution  of  the  following  problem: 

Find  g  e  W*(8)  so  that  for  all  h  e  W™(0), 

n  n 

Mg,h)  +  n  l  g(t  )h(t  )  -  n  l  z  h(t  ). 
k-1  k-1 

Proof.  For  g,h  e  define  I,  j  R  ♦  R+  ly 

L(uig,h)  “  LnJk(gtuh). 

If  gnj  exists  then  for  Vh 
(2.2,  0  -  ft  (0,gnX,h)  -  21(gnX,h),  -  2„-’  l 

so  it  is  necessary  that  be  a  solution  of  the  problem.  Furthermore, 

a2  .  n 

•=-r(u»g,h)  -  2Mh,h)  +  2n  l  h(t.r 
du  k-1  * 

n 

-  2 [ X  [  (D^.d'V).  ♦  n”1  l  h(t.  )2]  . 

|a|-m  0  k-1  * 

This  lest  expression  is  always  nonnegative,  and  equals  zero  just  in  case 
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(2.3) 


0%  »  0  for  V  o  e  such  that  |al  »  m, 
and  h(t^)  “  0  for  1  <  k  <  n. 

The  first  requirement  impliaa  h  is  a  polynomial  of  degree  <  m  -  1,  and  the  second 


implies  h  i  0  by  m-unisolvency  of  Hence.  Ln 


is  a  strictly  convex  function  on 


H^(fl),  and  any  (the)  solution  g  of  (2.2)  is  the  unique  minimizer  of  |_n^« 

He  now  show  that  L  ,  has  a  minimizer,  which  must  then  be  the  unique  solution  of  the 
nX  „ 

problem.  Consider  the  space  H  -  Rn  X  II  L  td)  (all  products  are  Cartesian  products) 

j-1  z 

where  N  •  } ,  equipped  with  the  Hilbertian  norm 


. V'S-”*1  J,  x  ^  'hj'o  • 


Define  the  linear  operator  U  :  w”(Q)  ♦  H  by 

U  a 

U  (h)  -  (hid  ) ,  D  ’h . D  A,), 

“  n 

where  a  .....a  are  the  N  distinct  elements  of  Z^  satisfying  |a|  •  m.  By  the 

argument  directly  following  (2.3),  U  is  injective .  He  claim  that  HQ=  Range  U  is  a  closed 

subapace  of  H.  This  will  complete  the  proof,  since  S^z  is  then  the  inverse  image 

under  U  of  the  orthogonal  projection  of  (z,0,0 , . . . ,0 )  e  H  onto  hq. 

Suppose  {h^:j  e  Z+}c  W^(Q)  is  such  that  {(Jh ^ }  us  a  Cauchy  sequence  in  H.  Now 

|*|  is  a  seminorm  on  wT(fl),  but  a  norm  on  (the  equivalence  classes  in)  w”(d)/P  , 

B  2  *  ffl 

where  Pm  is  the  space  of  polynomials  of  degree  <  «  -  1.  Note  that  H^tflJ/P^  is  a 

Hilbert  space  under  <•,•)_.  Thus  we  see  that  for  some  {p  }  C  P  and  h  e  H?(Q), 

n  j  —  * 

lh,  -  h  -  p.l  ♦  0.  since  U  is  continuous,  we  also  have  that 
3  I  * 

(h,(d  )  -  h(d  ))  -  p.ld  )  ♦  0  in  Rn. 

“j  n  ■*"  n  n 

But  by  assumption,  {h^(dn>  -  hfd^) }  converges  in  Rn  as  j  ♦  ",  so  that  ( A^)  [  does 

also.  Since  the  mapping  p  *  is  an  isomorphism  between  Pm  and  a  (closed)  subspace 

of  Rn,  it  follows  that  p.  ♦  p  e  P  as  j  ♦  Hence,  h .  ♦  h  +  p  in  W™(Q),  and 

j  »  3  2 

Uhj  *  U(h+p)  in  HQ,  showing  Hq  is  closed.  # 

Proposition  2.2.  Suppose  Assumptions  3  and  4  hold,  and  m  >  d/2. 

(i)  For  V  z  e  L2«J),  g^  ■  SyZ  exists  and  is  the  unique  solution  to  the  problem: 

Find  g  e  W*(fl)  such  that  for  all  h  e  H™(0),  Xtg.h)^  +  /  gh  dF  •  /  zh  dF. 


-9- 


(ii)  g^  is  the  unique  solution  to  a  boundary  value  problaa  of  the  for* 


Uf  (-4)  ♦  1)g  -  * 


B^g  “  0  on  38  for  1  <  j  < 


Here,  A  is  tha  Laplacian,  f  is  the  density  for  F,  and  the  Bj,  1  <  j  <  m,  are  linear 
differential  operators  of  order  a  +  J  -  t.  Furthermore ,  the  boundary  value  problea  is  a 
regular  elliptic  problea. 

Proof,  (i)  This  follows  by  an  argument  similar  to  the  one  used  in  the  proof  of  Proposition 
2.1.  Indeed,  the  arguaent  is  soaewhat  simpler  since  under  Assumptions  3,  and  4, 

X|h|2  ♦  fh2dr 

gives  a  norm  on  W?(8)  which  is  equivalent  to  1*1  (see  (2)  in  Theorem  4.2.4  of  t 1 7 ] ) . 

2  N 

Thus,  it  is  much  easier  to  show  that  the  appropriate  subspace  of  H  •  L2 '  L2*°*  is 

closed. 

(ii)  This  follows  from  standard  methods.  We  briefly  recap  the  argument  in  Agmon  [2), 
pages  141-143.  First  note  that  the  bilinear  form  B(u,v)  -  (u.v)^  is  uniformly  strongly 
elliptic  (Definition  7.1  of  [2]),  since 


and  for  all  (  e  I 


B(u,v)  »  l  (Dau,D°v) 
I«l*m  ‘ 


l  c2a  -  <  I  ;2)B  -  Hi*. 

#|«m  j-1  J  * 


Hence,  by  Theorem  10.2  of  (2],  there  exist  linear  differential  operators 
B^  «  B^(x,D) ,  x  e  30,  of  order  m+j-1  (1  4  J  4  a)  such  that 


8(u,v)  -  (»,AV)0  +  /fl  B^v  do  , 


where  A  -  (-A)®,  and  the  Bj's  are  nowhere  characteristic  for  3Q.  Here,  3/3n  denotes 
differentiation  in  the  direction  of  the  outward  normal  to  3Q,  and  do  is  the  element  of 
ares  on  38.  If  g  were  a  solution  to  the  boundary  value  problem  in  the  statement  of 


(11),  then  for  *11  h  *  H*(Q)  ve  hava 

Kg.h)^  ♦  /ghdF  -  X((-A)*g,h)0  ♦  /g(t)h(t)f (t)dt 
•  (f(t-g),h)Q  ♦  (fg,h>0 
-  (f*,h)Q  -  /shdF. 

Hence,  g  would  ba  a  solution  to  tha  variational  problem  atatad  In  part  (1).  Saa  alao 
Dions  *  Naganaa  [8],  Saotion  9.5. 

Ha  now  verify  that  tha  boundary  value  problaa  la  regularly  elliptic  by  checking  tha 
conditione  of  Definition  5. 2. 1/4  of  Triebal  (17).  That  If  1 (-A)*  la  properly  elliptic 
followa  front 

Xf(t)”1  I  C2*  -  If"1  l«l2®  >  0  for  all  C  e  »d\(0) 

l«l-m  tr 

and,  if  C,t)  e  ad  are  linearly  independent  then  the  roote  of 

P<»>  -  l  (t+tn)2o-0 
l«t-« 

are  tha  roote  of 

«  2 
I  (Ca+ttri^)  -  0 
1-1  3  3 

(replicated  m  times),  and  theae  are  clearly  complex  conjugatea  of  each  other.  Since  the 
boundary  operatora  {B^}  ^  are  nowhere  character latic  for  JO,  they  font  a  normal  eyatem 
and  furthermore  they  have  orders  <2»  -  1.  Finally,  that  Bj  satiafiea  the  complementing 
condition  with  respect  to  If  '(-A)*  ♦  1  follows  from  Remark  5.2. 1/4  of  (17).  Hence,  the 
problem  is  regularly  elliptic. 

To  complete  the  proof,  we  need  to  show  that  the  boundary  value  problem  possesses  a 
unique  aolutlon.  According  to  Theorem  5.4.3  of  [17]  and  tha  remark  thereafter,  it  suffices 
to  show  that  tha  only  solution  to  the  boundary  value  problem  when  s  =  0  is  the  trivial 
solution.  However,  this  is  lie  diets  since  If  1(-A)m  is  a  positive  operator  so  -1 
cannot  be  in  its  spectrum,  i.e.  Xf  ’(-A)"  ♦  1  has  empty  null  space.  • 


3.  Function  Spaces.  In  this  •action,  we  will  introduce  the  function  spaces  which  will  be 
used  in  the  sequel.  The  Sobolev  spaces  W^(Q)  were  defined  in  Section  1.  it  will  be 
useful  to  have  the  Besov  spaces  B*2<Q),  which  are  defined  for  all  real  s  >  0.  A 
complete  account  say  be  found  in  Tr label  [17].  The  following,  brief  definition  will  serve 
our  purposes.  Given  two  Banach  spaces  AQ,A^,  both  being  subsets  of  a  larger  Banach  space 
A,  define  the  K-functional  K:(0,«)  x  (AQ  +  A^ )  ♦  R+  (here,  AQ  ♦  A^  is  the  linear  span 
of  AgU  A^ ,  and  x  denotes  cartesian  product)  by 


K(X,a)  “  inf  { I •  ♦  lla1  1^  :  a  »  a0  ♦  a1  and  a^  e  A  ,i  m  1,2}  . 

A0  A1 

For  0  e  (0,1),  the  K-method  interpolate  is  the  Banach  space 


(Ao,A,)0,2 


ae  A„  +  A 


1* 


lal 


<Ao,Ai’e,2 


{/  [y'^IM))2)'1  dl}/2<  -}  . 
o 


One  of  the  basic  properties  of  this  interpolation  nethod  is  that  the  spaces  are  increasing 
in  0.  i.e.  if  0  <  0  <  ♦  <  1,  then  AQ  n  ai  C  (A^A^g  2  C  (Ag.A^  2  C  a,-*-  Aj.  For 
any  se(0,«),  let  A  be  an  integer  >  s.  Then  we  My  define  the  Besov  spaces  by 

B*2(0)  -  a2(0),  W^(Q))0  2  with  0-  s/k. 

The  definition  is  independent  of  V,  up  to  equivalent  renorming.  For  •  e  Z  ♦  , 

B22(Q>  -  w"(Q) 

with  equivalent  noma.  For  Q  -  Rd,  this  result  is  stated  in  Remark  2. 3. 3/4  of  [17],  and 
it  follows  for  general  0  from  Definition  4.2.1  of  that  reference.  Interpolation  of  Bescv 
•paces  yields  nothing  new,  i.e. 

<b*2(Q>,  b22(Q))0  2  -  B22-6),+  8r(n). 

This  follows  from  Theorem  4.3. 1/1  of  [17]. 

Let  Ma<Q)  be  the  subspace  of  W^"( 0)  of  functions  which  satisfy  the  natural 
boundary  conditions  B^h  -  0  on  90,  1  «  j  «  a.  That  these  are  well  defined  follows  from 
Sobelev's  trace  theorem  (Theorem  3.10  of  Agmon  [2]).  Also,  define  the  operator 

V  -  f"’(-A)* 
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with  domain  of  definition  N  (Q).  Here,  A  is  the  usual  Laplacian  on  Rd  and  f”1  denotes 

D 

division  by  ths  density  of  Fe  According  to  Theorem  14e6  of  Agnon  [2],  V  has  discrete 
spectrum  contained  in  the  positive  real  axis,  and  the  eigenvalues  have  finite 
multiplicity •  We  will  write  the  eigenvalues  as 

Y,  <  Y2  <  ... 

where  each  is  replicated  according  to  its  multiplicity.  The  corresponding  eigenfunctions 
will  be  denoted  and  we  may  assume  that  forms  a  complete  orthonormal 

system  for  Ljtfl)  (Theorem  16.5  of  Agmon  [2)1.  The  space  Nj,  is  given  the  following  norm 

Ml  m  5  *  (^1,lb-Vo 

m  v 

-  Iphl*  ♦  |h|*. 

One  notes  that  1*1  „  is  the  same  as  defined  in  1.18.10  of  Triebel  [17].  Using  the 

m 

definition  contained  therein,  we  may  define  for  any  nonnegative  real  number  s. 


n“(fl)  -  {he  L,(Q)  *  Ihl  5(1  <Y*/m  +  U(h,*  >*>  /z  <  -  > 

m  *  ..s,n»  v  v  v 


S  (8)  v 
m 


According  to  Theorem  1.18.10  of  [17],  if  6  e  (0,1), 


In  particular,  for  s  e  [0,2m] 


(L2(Q),  N>))9>2  -  N®*(8). 


a2(fl),NB(n)>s/2B,2  -n  (0). 


In  order  to  give  a  more  useful  characterization  of  N* ( 0)  it  is  necessary  to  introduce 

n 

Besov  spaces  with  boundary  conditions. 

Letting  be  the  boundary  operators  from  Proposition  2.2(11),  set 

P22  {b  l*0*  “  *h  e  B22,S)!BjhUn  “  0  f°r  **Ch  *  aatlsfying  m  ♦  j  -  1  <  s  -  y/2  ). 


(compare  with  section  4.3.3  of  [17]).  Note  that 


N  (8)  ”  (.  1<°>* 

m  m 
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Our  immediate  aim  ia  to  aatabliah  similar  equivalence  results  for  the  spaces 


N*(B),  s  «  l  . 

R  t 

Proposition  3.t.  Suppose  Assumptions  3  and  4  hold. 

(1)  If  s  e  R+  and  if  there  exist  no  1,  1  <  i  <  m,  and  no  j  e  Z+\{0)  such  that 

s  •  2m<  j-  V2  )  +  i  -  Vj  ,  then 


NS(B) 

ra 


Here,  the  set  of  boundary  operators  {c^}  are  the  class  of  all  operators  of  the  fora 
Cfci  -  Ba  o  (?k  for  1  <  i  <  m,  1  <  k  t  j  =  [s/2a]  +  1. 


(ii)  If  s  *  2m ( j  -  V2  )  +  i  -  V2  for  some  i,  1  <  i  <  m, 
«>>  ”  <h  e  B“2,{ckl>(a>  B«<Rd) 

(Hi)  For  all  s  e  *  the  norms  1*1  and  1*1 

b“2,q)  «•:<» 

particular,  for  k  e  Z  ,  the  norms  l«l  and  1*1 

n*(b> 


and  j  e  Z+  \{0 },  then 
with  supp  h  c  fl)  . 

are  equivalent  on  N8 ( B) . 

m 

]( 

are  equivalent  on  N  (Q). 

m 


In 


Proof.  Step  1.  First  assume  s  “  2mk  for  some  k  e  Z+.  We  will  prove  (i)  ani"  (iii) 
in  this  case  by  induction  on  k.  The  equality  in  (i)  is  true  for  k  -  1,  as  already 
noted.  Now  we  make  use  of  the  following  a-prlorl  inequality  in  Theorem  4.3.4.  of  117):  for 
any  p  e  Z+,  there  exist  positive  constants  C1 ,  C2  such  that  for  all  h  e  w2m+p(B), 


C,lhr  < 

1  2m+p 


I  Phi  + 
P 


Ihl  +  l  Ib  hi  ___  , 
0  J-1  3 


<  C  Ihl,  . 

2  2m+p 


Taking  p  *  0/  we  see  that  1*1  _  is  equivalent  to  1*1-  on  N*  ( ft) ,  since 

^(0)  2r • 


B  h  -  0,  V  h  e  N*  (0)  and  all  j,  1  <  j  «  m. 

3  For  induction  step,  assume  for  some  k  >  2  that  (i)  and  (Hi)  hold  when 

s  *  2m(k-1).  Then  h  e  N2"1'  (0)  implies  Ph  e  N2™"'  1 '  ( fl) ,  so  that  Ph  satisfies 
n  m 


for  1  <  i  <  a 


*1P<k”1>(Ph)  -  BiPkh  -  0  on  38 

.  Hence ,  all  h  8  satisfy  ths  boundary  conditions  in  (i).  How 

apply  ths  ar priori  inequality  aqain  to  conclude  that  l»l^v  is  equivalent  to  the  non 
(0>  given  by 

ll,h,2.(kM)  4  ,h,0  • 

but  by  induction  hypothesis,  Iflhl.  can,  for  all  h  8  N2ak(  0) ,  be  trapped  between 

constant  nultlplaa  of 


I  Phi 


jj2e(k-1 ) 


(0) 


(lP<l,_1)(ft>)l2  +  iPh  I*  y* 
(iP^'o  + 


By  using  the  eigenfunction  expansion  for  h,  ws  easily  see  that 


T^IhlJ  <  ItfclJ  <  y"2<>t’1,lP,thlJ  . 


It  is  now  clear  that  I  •  is  equivalent  to  the  non  given  by 

(iP^hl*  ♦  lhl*)V2-  Ihl  . 

u  (£**<8) 

This  shows  that  (i)  and  (iii)  hold  when  s  “  2mk  for  soee  k  e  Z+. 

8tsp  2.  Parts  (1)  and  (ii)  ars  now  an  mediate  consequence  of  Step  1  and  Theoreat 
4.3.3.  of  [17].  Part  (iii)  follows  fro*  the  non  equivalences  of  Step  1  and  a  simple 
calculation  with  ths  definition  of  the  K-wathod  of  interpolation.  t 
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4.  The  Green's  Function  Approximation.  The  sain  result  of  this  ssetion  is  Theorem  4.3, 
which  gives  an  approximation  of  the  smoothing  spline  in  terms  of  the  family  of  Green' a 
functions  desesribed  in  the  introduction.  The  proof  of  this  theorem  depends  on  the  two 


technical  leasws  (I 


4.1  and  4.2).  The  first  provides  various  norm  estimates  on  the 


family  of  Green's  functions.  The  second  lemsM  concerns  the  accuracy  of  approximating  the 
inner  product  in  LgCT,,)  by  the  inner  product  in  an<J  in  particular  if  one  of  the 

entries  in  the  inner  product  is  from  the  Green's  function  family. 

Since  Sjt  is  the  solution  of  a  certain  boundary  value  problem  (Proposition  2.2 (ii)), 
it  has  the  representation  given  in  (1.3),  where  G^( •, •)  is  the  Green's  function  for 
1P+  1  with  dosMin  Ha(0).  We  prove  this  as  in  Theorem  5.6.4  of  Triebel  [17]  by  giving 

an  explicit  construction  for  G^.  Let  »  (4V)  be  the  eigen system  for  the 

operator  V  “  f_1(-A*)  with  domain  Nb(Q)  as  described  above,  ttien  we  may  develop  Gx  in 


a  bilinear  expansion,  viz. 


G.(t,r)  -  l  (1+Xy  f1*  <t)*(T>, 


where  the  series  converges  (at  least)  in  Ljtfl  x  0).  The  aforementioned  reference 
guarantees  G^  6  W2P  (fi  x  Q)  for  any  p  e  Z+  satisfying  p  <  2m  -(d/2)  (and  not  for  any 
larger  p). 

we  now  Introduce  some  notation  used  in  the  segue 1.  For  i,  M  Z^,  define 

G  ®,B  ( t , t)  -  d“  dJ  Gx(t,T)  , 

where  Dt  denotes  differentiation  with  respect  to  the  first  d-dimensional  argument,  and 
similarly  for  Df.  Also,  we  shall  use  the  notation  for  arbitrary  nonnegative  functions 
g,h 

g(t)  ■  h(t) 

to  mean  there  exist  positive,  finite  constants  C^,  Cj  such  that  for  all  t  of  interest 

C^t)  <  h(t>  <  C2g(t)  . 


The  notation 


g(t)  ~  h(t)  as  t  ♦  t. 
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Mans  that 


g(t)  -  h(t)  “  o (h(t) )  as  t  ♦  tQ  . 

Finally,  we  shall  need  the  ltm  aerator  R  .  :  c(8)  ♦  w”  (8)  given  by 

n  A  2 


<R  .h)(t)  -  /  G.(t,T)f(T)_1h(T)dF(t)  -  n'1  £  G.(t,t  )f(t  )"’h(t  ) 


-1 


'nV 


k-1 


Lemma  4.1.  Let  2m  >  d/2, 
(1)  If  0  <  p  <  2m  -  d/2, 


»  /  G^(t,x)f ( T)~*h( T)d(F(  t)  -  P^( t) ]  . 
and  let  Assumptions  3  and  4  hold, 
then 


'^(QxQ, 


x-(2p+d)/4m 


Here  the  constants  may  be  chosen  so  as  to  depend  only  on  8,  m,  p,  and  F. 

( ii)  If  p  e  and  p  <  2m-d,  then  G^  H  <3^(8  *  8) ,  the  class  of  functions  whose 
derivatives  of  order  <  p  are  bounded  and  continuous  on  8  x  Q. 

(iii)  If  p  <  2m-d,  then 

-<p+d)/2m 

sup  IG,( «,t)l  <  K  \ 

A  P  3 


where  K,  «  K  (8,m,p,F).  Here  IG, ( * ,t)  I  Mans  1*1  computed  when  G,  is  considered 

5  A  p  p  A 

as  a  function  of  its  first  argument  only. 

Proof,  (i)  Rather  than  estimate  IG.I  ,  we  work  with  the  equivalent  norm 

w^ta  x  a) 


III  0.  lit  as  defined  in  (1.2).  Since  Pis  formally  self  adjoint, 

<(8x8) 

Gx<t,T)  -  Gx(T,t), 

and  we  clearly  have 


(4.1) 


I  I lGxl | | 


2 

W^(8x8> 


lr  I * 

b\L  2(flxfl) 


2  /  ll|GxC,t)|||*dT  , 
n  a  P 


where  by  gx<’,t)  we  mean  to  consider  Gx  as  a  function  of  its  first  argument  while  the 
second  is  held  fixed  at  t.  By  Proposition  3.1,  there  are  positive  finite  constants  c, 
and  Cj  depending  only  on  8,m,p,  and  f  such  that  for  every  t 
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(4.2) 


C.'G.I-.T)'  <  II|Gx(*.t>|||2  <  c  IG.e.T)! 

1  *  NP(fl)  *  p  2  X  f(at 

■  B 


According  to  Thtom  14.6  of  Agmon  [2],  we  hav. 

(4.3) 

We  not.  that  0  need  only  satisfy  the  restricted  cone  property  in  order  for  this  to  hold 


2m/d 

Y  *  V 

'v 


(...  p.  239  of  (21).  Hence,  for  some  constants  -  c^tfl.m.p.F),  1  -  1,2,  it  holds  that 


2  . 

/  IS C,t)l  dr  -  l  (1  +  XT  )  YV 

A  HP(Q)  v  v  v 

m 

<  c,  l  (1  ♦  c2xv2-/d)-2v2*>/d 

V 

~  r  %“(2p+d)/2»  r  x2p/ddx 

l  (HC^V 

as  A  +  0.  T*e  last  line  follows  from  an  application  of  dominated  convergence.  This 
show,  that  for  some  finite  constant  Kj  *  K}(0,m,p,F), 


IG.I  <  K  .-Up*d)/4m  ' 

*  W^(B  x  0)  1 


An  entirely  similar  argument  shows 


IG.I 

x  1^(0 


x  fl) 


>  K2X-<2p+d)/4- 


for  some  constant  K2  ■  K2(Q,m,p,F).  This  completes  the  proof  of  (i). 

(11)  8ince  Gj  e  W2(0  x  0)  for  any  q  <  2m  -  (d/2),  it  follows  that  G^  e  C^(0  x  0)  for 
any  p  <  q  -  (d/2)  by  one  of  Sobolev's  imbedding  theorems  (see  Theorem  5.4,  part  (C)  in 
Adams  (1) ) . 

(ill)  Considering  lG. (*,t)l  as  a  function  of  t,  w.  have 

*  Np(0) 

m 

(4.4)  lG.C,t)l  -  l  (1+Xy  )  ■J  if'*  I.  .,<t>|2 

A  HP(B)  V 

■ 
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where  the  aariaa  converges  (at  laaat)  in  1.^(0),  provided  of  couraa  that 
p  <  2a-  d/2  .  Tha  a  priori  aatiaata  (Thaoran  5.3.4.  of  Triabal  [17))  yields 

'♦v'2»  ‘  C3(,0*v'o  +  '♦v'o>  «  VYv  *  11  '♦v'o  “  C3(1fv  +  %) 

where  the  constant  C3  *  c^lfi^n.P).  By  Sobolev's  inequality  (p.  32  of  agaon  [2])>  for  soaw 
“  c4(Q,e)  and  any  r  >  1,  we  have  for  all  t  e  Q 


IVOI  <  C4(r"(2“"d/2>  l*vla«+  r4/*Uv,#* 


<  C4  <C3  <VD  r-'2-^25  *  r^2  ,  . 


If  we  now  utilise  yv  “  v2*^4,  and  put  r  ■  then  we  Obtain 

U  (t)|  <  C5v1/d  for  soae  C5  “  C5  (Qre) .  Substituting  this  back  into  (4.4)  yields 


IG.  (*,t)l2  <  C I  l  v(UW  )"27uP/* 

*  N?(0>  5  v  v  v 


,2*/dl-2w(2ptd)/d 


<  C2  I  (l-K^lv*'  ) 


~c2c, 

5  1  0  (14C3S2*/d)2 


as  140.  Note  that  convergence  of  the  latter  Integral  requires 

P  <  2n  -  d. 

In  view  of  Proposition  3.1,  the  proof  of  (iii)  is  coaplete. 
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Lemma  4.2. 

Let  m  >  3d/2,  and  let  Assumptions  3  and  4  hold. 

(i)  For  functions  h,g  e  Wd(Q)  we  have 

1/  hg  d(F-Fn)|  4  K4dnlhldlgld 

where  K.  »  K. (0)  Is  a  constant,  and  d  “  sup|F-F  |. 

4  4  n  n 

(11)  Suppose  p  e  Z+  satisfies  p  <  2m  -  (3d/2).  Then  for  any  h  e  Wd(B), 


I R  .hi 
nX  p 


Kid  IG.I  .  _  Ihl 
4  n  x  wf  d(axn)  d 


where  C,  where  C  depends  only  on  F,  and  X4  Is  the  same  constant  as  In  (1). 

(Ill)  Under  the  same  hypotheses  a*  In  (11),  for  each  v  e  Z+, 


<xh,P 


4  vW5d/4B)W 


■(p~d)/2m 


Ihl . 


where  «  K^Q.rn.p.F) ,  1  «•  5,  6,  are  constants, 

(lv)  Under  the  same  hypothese  as  In  (11),  there  Is  a  constant  ■  K7(Q,m,p,F)  such  that 


-1 


l 

k-1 


*9 


Xk  p 


Ig.l.  S  M*<Pt2dl/2,(Hd  X_d/") 
Xk  d  7  n 


where  g^  Is  defined  In  (1.5). 

Proof.  (1)  We  start  with  the  following  Integration  by  parts  formula,  valid  for  any 

m  <J 

h  <  Cg  (I  )  *»  infinitely  differentiable  functions  of  compact  support,  and  any  probability 

distribution  function  G  on  Bs 

(4.5)  /  h(t)dG(t)  -  l  (-1)|B|  /  D$h(t[M  )*G(t(61)dt 


S{o,i}a 


where  t[6)  has  )th  coordinate  t^(81  given  by 


tjfW 


if  -  1  , 
if  b3  -  0  , 
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where  A  >  0  ia  chosen  so  that  ft  C  (-A,A]d  holds.  The  summation  In  (4.5)  Is  over  all 
S  e  zj  whose  coordinates  are  either  0  or  1.  The  Integrations  are  over  [-A,A] d.  Note 
that  each  integral  on  the  right  hand  aide  may  be  reduced  to  a  | B!  -dimensional  integral 
with  respect  to 

n  dt  . 
jsSj-1  3 

A  proof  of  (4.5)  runs  as  follows.  If  G  is  a  unit  point  mass  concentrated  at  some  point 
in  Q,  then  (4.S)  can  be  proved  by  a  tedious  but  straightforward  induction  on  d.  The 
result  for  arbitrary  descrete  G  follows  by  talcing  convex  combinations.  It  then  suffices 
to  show  that  if  the  formula  holds  for  each  element  of  a  sequence  G^,G2, ...  which 
converges  to  some  G  uniformly  in  ft  ,  then  it  holds  for  G,  because  the  discrete 
probability  measures  are  dense  in  the  space  of  all  probability  measures  on  [-A,  A]d  when 
equipped  with  the  topology  induced  by  Kolmogorov's  (sup-norm)  metric.  Since  Kolmogorov's 
topology  is  stronger  than  the  topology  of  weak  convergence,  we  Immediately  have  that 

( 4.6)  /  h(t)dGn(t)  *  /  h(t)dG(t )  . 

Furthermore,  if  Be  {o,l}d  we  claim  that  as  n  • 

/  D®h(t(BJ  )G  (tCBDdt  ♦  /  D^i(t[BJ  )G(t( B)  >dt  . 
n 

^  4 

Since  distribution  functions  are  bounded  by  1  and  h  e  C0<R  ),  this  latter  result  follows 
by  Lebesque's  dominated  convergence  theorem,  completing  the  proof  of  (4.5). 

To  complete  the  proof  of  (i),  assume  first  of  all  that  0  -  [-A,A]d  and  also  that 
h,g  e  CQ (Rd) .  Then  the  product  differentiation  rule  followed  by  Cauchy- Schwarz  yields 

(4.7)  )/  1  (F  -  Fn)D®hg)}(tlBl  >dt|  <  dnJ  |DS(hg)(t(B)  >  I  dt 


<  d  E  /  I  {(D^Sj)  (D®~ag)  }(t[  B]  )  Idt 
CtfB 

<d  E  [/  (Dah)2Ct[B])dtj/2  [/  (De’ag)2(t[B])dtl/2  . 
"  (KB 
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If  |0|  <  d,  Sobolev's  theorem  on  traces  (p.  38  of  Agmon  12])  applies  and  we  have 

(/  (D“h)2(tl01  )dt)1/z  <  C  Ihl  . 

1  Wd(l-A,A]d> 


where  ( A)  is  a  constant.  Hence,  from  <4. 5)  and  (4.7)  we  obtain 

(4.8)  1/  hg  d(F  -  r  )|  <  22d  max  (l,C.  }d  Ihl  .  .  !gl  .  .  . 

”  1  "  WdU-A,Ald)  Hd((-A,A]d) 

m  (J  d 

This  was  obtained  under  the  assumptions  that  h,g  e  CQ(R  )  and  8  -  1-A,A]  .  However, 

the  restrictions  of  ( Rd )  functions  to  l-A,A]d  gives  a  dense  set  in  wd( [-A,A] d) ,  so 

d  d 

(4.8)  holds  for  arbitrary  h,g  e  (-A,AJ  ).  Furthermore,  there  exists  a  continuous 
extension  operator  (Theorem  4.32  of  Adams  Ml)  E«  wd(8)  ♦  Wd([-A,A]d),  so  that 

Hd([-A,A]d)  norms  in  (4.8)  may  be  replaced  by  Wd(8)  norms  at  the  cost  of  introducing 
another  constant  factor  which  depends  on  0.  this  completes  the  proof  of  (i). 

(ii)  Define  G^(t,x)  -  G, (t,x)/f(T).  The  assumptions  on  p  guarantee  that  G*'°  is 
bounded  and  continuous  on  8  x  8  by  Lemma  4.1  (ii)  and  Assumption  3,  provided  |a|  <  p. 
Hence,  we  may  interchange  differentiations  and  integrations  under  these  conditions.  Thus 

(4.9)  iR  .hi2  -  1/  G.C,x)h(x)  d[F(x)  -  F  ( x)  J  I2 

nA  p  a  n  p 


-  I  1/  G?'°(*,x)h(x)  d[F(X)  -  F  (x)]|2 
|o|<P  n 


<  K*d'lhr 
4  n  d 


I  / 

|a|«p 


(t,«)  r  dt 

<3 


where  the  last  line  follows  from  part  (i).  Now 


I  j  IG?'°<t..)l2  dt  -  J  J  /  /  |Gd'B(t,X)|2  dtdX  4  . 

lo|«p  |a|<p  I0l<d 

An  application  of  the  product  differentiation  rule.  Assumption  3,  and  Sobolev's 
inequality  yields 
(4.9a) 

where  C  is  a  constant  depending  only  on  f.  Substituting  this  back  into  (4.9)  completes 
the  proof  of  (11). 


•Vptd  «  C  iG^d, 
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«- 


(ill;  Using  psrt  (ii)  and  Lease  4.1  (i)  gives  for  any  v  e  Z+\{o}, 


IRV.h«  <  K^cl  IS.1 

nX  P  4  n  X  nX  d 


for  ease  constant  -  C^fl.a.p). 
gives  the  desired  result. 


Using  p  •  d  in  this  last  inequality  and  iterating 


(iv)  We  have  by  Cauchy-Schwars 


(4.10) 


-1 

n 


1  'Vp^Xk'd  -  /  ^’'‘"pV^'d  dVt> 

k*1 

<  [/  lGAC,t)lp  dPn(t)f/a  [/  !3xC.t)l*  dFB(t)J/2. 


Mow  concentrating  on  the  first  factor,  we  obtain  from  part  (i)  that 


/  lG.C,t)lJ  dr  (t)  <  /  lG.C,t)l*  <tf(t>  +  MB.,2  . 

1  x  p  "  x  p  4n  *  w^d(axfl) 

Mow  lima  4«1f  inequality  (4»9a)i  and  Asaunption  3  imply  the  existence  of  constants 
Ci  "  cA(0'*'P'y)'  1  -  ^ 2,  for  which 


/  !Gx(«,t)i*  dr(t)  <  c1x"(2p+d)/2“ 


and 


Bence 


<  e  x"(2P+3d,/2“ 

X'wTd(Rxfl)  2 


IG,I2 


/  .Gxe,t).2  «B(t)  <  ClX-<2^d,/2"  ♦  K4C2dnX-(2P+3d)/2“ 


If  we  set  p  -  d  in  this  latter  result,  and  substitute  the  bounds  back  into  (4.10),  then 
the  desired  result  is  obtained. 


* 


-23- 


Theorem  4.3.  Suppose  m  >  3d/2,  and  that  Assumptions  2,3,  and  4  hold, 
(i)  There  exists  nQ  «  n0(fl,m,  {Xn}  ,F,  {d^J )  such  that  for  all 
n  >  n^  and  all  j,  1  <  j  <  n. 


’nXj 


v=0 


RnXgXj 


where  the  series  converges  in  W^(fl)  for  any  p  e  Z+,  p  <  2m  -  3d/2.  Moreover,  for  any 
fixed  n  >  nQ,  this  convergence  is  uniform  in  j  (1  C  j  <  n)  and  X  e  [X^,*). 

(ii)  There  exists  n.  «  n, (8,m,{X  })  such  that  for  all  n  >  n  and  any  p  e  Z., 
p  <  2m  -  3 d/2, 


Hg  .  -  g.  I  <  K  (d  \_5d/4m)  x“<p-d)/2m 

nXk  Xk  p  7  n  Xk  d 


where  »  K^(Q,m,p,F)  is  a  constant. 

Proof ,  (i)  For  any  fixed  tQ  e  fl,  let  G^(t,T)  »  G^(t,  t)/f  ( T)  as  before,  and  put 

*xo  -«x(*'V  • 

If  v  e  Z+  and  0  4  p  <  2m  -  3d/2,  then  Lemma  4.2  (iii)  yields 


(4.11) 

Hence,  if  only 

(4.12) 


lRV.g,n|  <  K  (K  d  x’5<J/4m)W  X"(p_d)/2”lg  I 
nX  XO  p  5  6  n  AO  d 


d  x  "5d/4B  <  k:1 

n  n  6 


then  the  series  in  the  statement  of  the  Lemma  converges  in  W^((2).  By  Assumption  2,  there 
is  an  nQ  *  nQ (Q,m, {Xn} ,F, td^J )  for  which  (4.12)  holds  for  all  n  >  nQ.  Lemma  4.1  (iii) 
and  (4.9a)  give  the  following  bound,  independent  of  tQ: 

•Wd 4  V'd/B' 

which  implies  the  convergence  of  the  series  uniformly  in  j.  Substituting  this  latter 


inequality  back  into  (4.11)  and  putting  C4  *  KjK3  yields 


(4.13) 


iR^g..!  <  c  (K  d  x'5d/4m)v  X_(P+d)/2m 

nX  XO  p  4  5  n 


which  implies  the  convergence  uniformly  in  X  >  X 


-24- 


r 


To  complete  the  proof  of  (i),  we  need  only  show  that 
(4.14)  h  =  y  -v 

h '  i o  nX  * 


is  in  fact  equal  to  gnXfc.  In  order  to  do  this ,  we  will  first  show  that  for  all 
X  e  w”(B),  and  any  fixed  to  tg  e  Si, 

(4.15)  X(Gx(t0,*).x)B  +  /  Gx(t0,‘)x>iF  -  X<tQ>  . 

we  claim  that  N2“(fl)  is  dense  in  W?(G),  so  that  is  suffices  to  prove  (4.15)  for 
m  2 

x  e  N20.  This  density  claim  follows  since  the  linear  span  of  the  eigenfunctions  (4^ 
m 

is  dense  in  both  N20  and  NB,  and  the  latter  is  the  same  as  W?  but  with  an  equivalent 
in  m  ^ 

norm  (Proposition  3.1).  Now  for  x  e  N^“,  “•  “V  “PPly  the  Green's  formula  and  note  that 

the  boundary  terma  vanish  to  obtain  that  the  left  hand  side  of  (4a 15)  is  equal  to 

/  {xmf1  ((-a)bx) (t)  ♦  x<t)>  g  (t  ,t)dT  . 
n 

However,  this  last  quantity  is  equal  to  Xttg)  by  the  definition  of  as  a  Green's 

function. 

Now  note  that 

*  «• 

h  -  »Xk  *  J^x’lk  *  S»X  J0£x9Xk  “  RnXh  ' 

where  the  second  equality  is  justified  (if  m  >  3d/2)  by  the  fact  that  the  series  converges 
absolutely  in  wB(0),  and  R^x  is  a  continuous  operator  on  WB(fl).  Furthermore,  using 

a 

Fubini’s  theorem,  we  obtain  for  any  x  e  W2(Si)  that 
X(RnXh'K>m  +  ^(RnXh)xdF  “ 


-  X  [  /l  /G?'0(t,T)h(T)d(F(t)-F  (T)]  }(D°x)(t)dt 
|a|-m  A 


+  /  {  /  Gx(t,t)h(t)d(F(T)  -  Fn(t)]  }x(t)dF(t) 

-  /  {l  l  \  G^'°(t,T)(D°x)(t)dt  +  /  Gx(t,T)x(t)dF(t)}'h(T)d[F(T)-Fn(T)  ] 

|a|«m 

-  /  x(T)h(T)d(F(T)  -  F _(T)]  , 

'  n 
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where  the  last  line  follows  frost  (4.15).  These  last  two  sets  of  displayed  calculations  can 

now  be  used  to  show  that  for  any  x  e  w“(0), 

X(h,x)  +  /  hxdF 
n  n 

*  i{9Xk'x)«  I  9lkX  **  +  X(Rnlh,X,m  +  ^  <RnXh>x  dr  "  /  hx  dlr“FnJ 
-  x(tk)  . 

where  the  last  line  follows  frost  (4.15).  It  now  follows  from  Proposition  2.1(1)  that 
h  ■  1n^k>  hud  hence  that  the  proof  of  (1)  Is  ccsiplete. 

(11)  If  n  >  nQ(  we  have  frost  (4.11)  that 


*9nXk  -  *Xk'p  ‘  1  ,RnVx* 


v-1 


VI 


where  Cg  ■  C^tQ.e.p.P).  If  we  take  **  (fl,», {A  }f {dn)>  sufficiently  large  that  for 


all  n  >  n^ 


then  part  (11)  follows. 


d  X  “5d/4*  <  (2K 
n  n  D 
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5.  Main  Theorem,  The  results  of  tha  previous  sectiona  ara  brought  togather  hara  to  prove 
the  following  aain  result. 

Theorem  5- 1 ■  Suppose  Assuaptions  1  through  4  hold#  and  that  a  >  (3d/2).  tat  p  e  Z+# 
with  p  <  2a  -  (3d/2)  . 

Suppose  g  e  Nq(Q)  where  q  satisfies 

d  <  q  <  2a  +  p,  p  <  q  • 

If  g  «  S  .  *  with  the  observational  aodel  (1.1),  then  as  n  ♦  • 

Sl,nA  -  ,.2  -  0[x(q_p,/*  ♦  n"*  X~ ^ 2p*d)/2a ] 


uni f oral y  in  X  e  (X  ,A  1 . 

n  n 

Remark  5.2.  The  last  phrase  aeana  there  exists  an  nQ  (depending  only  on  B,a,p,q,  {Xn>, 
(An),  and  (dn) )  such  that  there  is  a  constant  C  for  which  n  »  nQ  implies  for  all 
X  6  # 

Big,,  “  ««£  4  '[X^"  *  n-’x-<2p+d)/2-]  . 


Pffnrt  5.3.  Froa  the  two  lemaas  below,  we  see  that  a  somewhat  sharper  result  can  be  stated 
if  one  is  willing  to  use  the  equivalent  N^(Q)  norm,  naaely,  for  all  6  >  0, 

sup  Big  .  -  gl2  ~A  (X)92  +  /  «6.(*,t)l2  dF(t)  . 

Ig  lq  -  9  "X  1^(0)  **  *  N^CO) 

n 

Here,  A  i  R  ♦  R  is  given  in  (5.2),  and  satisfies  the  relation  a  ( X)  ■  X<q  p)/,a. 

pq  +  +  pq 

The  supremua  is  taken  over  all  g  e  Nq(fl)  for  which  Igl  -  9. 

"  l£(B> 

Proof  of  Theorem,  Using  Assumption  1,  we  calculate 

B,*nX  '  *'p  "  B'SnXS<  V  *  £  -  »'  p 

“  '^‘V-S'p  +  2n_1  l  (Eek)  ^nX*1  V'*'*nXk  >p  +  B,Snx£'p 

-  «snAs<V  -  ♦  XiSn)t I2  . 
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Thia  ia  tha  familiar  decomposition  of  Man  squared  error  into  biaa  squared  plus  variance. 

The  theorem  ia  now  an  immediate  consequence  of  the  following  two  lemmas  and  the  equivalence 

of  l*lp  and  I'l^p^  (Proposition  3.1). 
m 

Leansa  5,4.  Under  Assumptions  2,3,  and  4,  if  m  >  (3d/2),  p  e  Z+  and  q  e  (d,»)  satisfy 
p  <  2m  -  ( 3d/2)  and  p  <  q  <  2m  +  p,  then  for  every  9  >  0, 


■up  .g(A  )  -  gl2  :  g  e  Nq(fl)  and  Igl  »  9j  ~  A  (X)92 

1  n  A- ■  n  1m  rt  #  r\/i 


Nq(fi) 

n 


pq 


as  n  *  •.  Here,  the  function  A^  ia  determined  by  p,q,Q,m,  and  P,  and  furthermore 

A  (X)  -  X(q-P)/B  . 

pq 

For  both  asymptotic  relations,  there  exists  n  -  n  (Q,m,p,q,{X  ,A  },{d  },  f)  such  that 

0  0  n  n‘  n 

for  all  n  »  n  ,  the  relations  hold  uniformly  in  X  e  [X  ,A  ]. 

o  n  n 

Proof.  First  note  that 

SnX*(V  “  Jf  q<VqnXk  *  \  C  /G  (t,.)g(t>dFn(t)  , 

Jc-1  u«0 


so  that 


Sn*a(V  '  q"  tJ0RnX  (/  Gx(T,*)g(T)dF(t)  -  l^x,)  ]  -  g-  JqCcX 


where 


eX  “  f  G^(  •,T)g(t)dT  -  g 


In  order  to  show  convergence  of  the  series  and  obtain  bounds  on  the  terms,  we  need  to 


estimate  e , .  Now 


ex“  I  ♦  *V‘1xV*'Wv  ' 


V*1 


where  {y  },  {♦  }  is  the  eigensystem  for  V  Introduced  above.  He  see  immediately  that 


(5.1) 


U.12 

NP(S1)  v-1 


I  (i  ♦  vp/"(1  ♦  xV’2uV2<g'Vo  4  AM(X"9|  l 

n  Nq(0) 
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where 


(5.2) 


a  (A)  -  sup  (1  +  r  >(p”q)/“(i  ♦  Ar  )"2(Ay  )2 

M  v>0 


Moreover,  if  the  sup  is  attai.ud  at  some  (finite)  vQ,  then  the  upper  bound  in  (5.1)  is 

attained  at  any  scalar  multiple  of  4  .  Utilising  the  eigenvalue  estimate  (4.3)  gives 

0 


(5.3) 


A  UXC,  sup  v2(p-q)/d(i  +  2m/d)uv2-/d)2 

P*  2  V>0  1 

4  C,A(q-P)/B  sup  (1  ♦  C,  xf2x(2**P"*,/B  . 
—  * 


x>0 

where  -  C^(fl,m,p,q,F) ,  i  “  1,2.  are  constants.  Here  we  have  replaced  the  discrete 

a_  /a 

variable  Av  w  by  the  continuous  variable  x.  Now  if 

p  <  q  4  2m  +  p 

then  as  x  ♦ 


(1  ♦  C1x)-2x(2“+P-<3)/B  ♦  0  , 


and  remains  bounded  as  x  ♦  0.  Alls  shows  that  the  sup  in  (5.2)  is  attained  at  some 
Vg,  and  hence  that  the  upper  bound  in  (5.1)  is  attained.  Note  that 
(1  ♦  CjX)  V  *■*"*>*  remains  finite  if  only  q  >  p.  Furthermore,  one  can  obtain  a 
lower  bound  tor  A^(  A)  which  is  of  the  same  form  as  (5.3)  (only  the  constants  are 
different) ,  and  so 

A  <A>  -  . 

W 

m 

To  complete  the  proof  of  the  beams,  we  need  only  show  that  £  l^.c.  is 

v-1  ”A  A 

asymptotically  negligible  compared  to  c^.  He  apply  Lemma  4.2(iii)  to  obtain  for  all 
v  >  1 

,RnAeA'p  ‘  C3<C2dnX"5d/4")V<P'<,>/2*'eA»d  • 
and  hence  that  for  all  n  sufficiently  large 

1  I  rVi*i'  *  A"5d/4B)l(<S’p)/2Ble.l 

vt  "u  P  n  *  d 

<  X '  Igl  (d  A_5d/4m)  xld_P)/2m  x(q-d)/2m 
q  n 


4  1C*  Igl  (d  x~ii/*K)X,‘<rp)/2m 
q  n 
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Assumption  2  la  used  at  tha  laat  step. 


Turning  to  tha  second  tarn  in  (5.4),  an  application  of  Cauchy-Schwars  gives 

1  a"1  l  <*Xk'  *Xk  "  »nlk  >p  1  *  n‘1  l  *«Xk,p,«Xk  -  Wp 

,  .  .-5d/4a. .-(p»d)/2a  -1  r  ,  ,  ,  . 

<  VV  >A  n  I  •9ik«p»9Jk'd 

where  Thaoraa  4.3(ii)  waa  used  at  tha  laat  atap.  Now  by  Lana a  4.2(iv), 

x-(p-d)/2n  #-1  £  {1  ♦  dttX’d/*)0(X-(a^,1,/2-)  . 

Combining  tha  laat  two  displayed  aatiaataa  and  using  Assumption  2  gives 


(5.7) 


“"1  J  <  *Ak'«Ak  “  «nlk  >p 


0(X-(2ptd,/2.)  . 


Finally,  tha  third  tarn  in  (5.4)  ia  aaally  daalt  with  in  tha  aaaa  aannar,  and  ona 
obtains 

<5.8)  n"1  I  l,x|c  -,nXk.a  -  (dnX'5<1/4-)2  (X-<2^d,/a-,  -  oU-,2p*d>/2*>  . 

Combining  (5.5),  (5.6),  (5.7),  and  (5.B),  and  inserting  than  into  (5.4)  yialds 
*lSnXel2  -  oV1  /  ISx(»,t)l2dF(t)  -  n-’x-(2p*B>/2n  _ 

Tha  atataaant  regarding  uniformity  follow  from  tha  corresponding  stataaants  in  Lamas  4.1 
4.2,  and  Thaoraa  4.3. 


Bmarks  5.6.  throughout  those  r marks,  we  adopt  tha  notation  and  hypotheses  of  Thaoraa 
5.1. 

(i)  Tha  beat  possible  rata  of  convargencs  implied  by  Thaoraa  5.1  is  always  obtained  if 

x  .  B-2m/(2gtd) 


in  which  case 


*,gnX  -  glj  -  0{n”2**"p^2<I*d* }  . 
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As  will  ba  noted  in  (vi)  below,  implicit  here  are  certain  lower  bounds  on  q  besides  these 
stated  in  the  theorem.  We  proceed  to  analyse  some  of  the  various  cases  with  regard  to 
smoothness  of  g  and  boundary  conditions. 

(ii)  If  q  <  m  +  V2  ,  then  N^(Q)  •  B^  (0) ,  and  there  is  no  difficulty  with  boundary 
conditions. 

(iii)  If  m  +V2<  q  <  3m  +  V2  ,  then  necessarily  B^g  ■  0  on  38  for  all 

k  e  Z+,  1  <  k  <  m,  such  that  m+k-1  <  q  -  V2  (see  Proposition  3.1).  Por  a 
particular  p,  the  requirement  q  <  2m  +  p  may  be  the  limiting  factor.  In  particular,  if 
p  *  0,  then  there  is  no  gain  in  convergence  rate  for  q  beyond  2m.  This  is  an  example  of 
the  saturation  phenomenon  familiar  in  approximation  theory.  Note  however  that  we  can  still 
make  gains  in  estimating  higher  order  derivatives  (p  >  1). 

(iv)  If  3»+V2<q<2m+p<*m  -  (3d/2),  then  not  only  does  g  satisfy  the  natural 

boundary  conditions  B^g  -  0  on  38,  1  <  k  <  m,  but  also  seme  second  order  natural 
boundary  conditions,  namely  B^g  *0  on  38  for  k  such  that  3m  +  k  -  1  «  q  -  • 

This,  of  course,  only  has  an  effect  on  estimating  derivatives  of  order  m  +  1,  m  ♦  2,  ..., 
[2m  -  (3d/2)] . 

(v)  Note  that  when  q  *  m  +  V2  ,  the  assumption  g  e  N^(8)  limits  g  in  both  smoothness 

and  boundary  conditions.  If,  for  example,  g  e  W^"(8),  but  for  sosm  j,  1  <  j  <  m,  we 

have  B  g  -  0  on  38  if  k  <  J  but  B.g  t  0  on  38,  then  g  e  Nq(8)  for  all 
k  j  n 

q  <  m  ♦  j  -  V2  ,  but  no  larger  q.  Hence,  we  have  for  every  e  <  0  that  the  rate 

B.g„x  -  -  0[n2(m*  J- V«>/<2-2j-1-2.»d)} 

is  obtainable.  It  would  be  interesting  to  sharpen  this  result,  by  going  to  logarithms  for 
example. 

(vi)  In  order  to  verify  Assumption  2,  it  is  necessary  to  know  how  fast  d  •  sup|F  -  F  I 

n  n 

can  go  to  sero.  For  d  -  1  and  Q  «  (0,1),  one  can  easily  check  that 

d  >  <2n)-1 
n 

with  equality  just  in  case 

-  (k  -  V2  >/n  1  <  k  <  n  . 
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For  gmnmral  d,  if  n  -  j  for  sool%  j  e  Z+,  than  tha  uniform  discrata  maaaura 
concantratad  on  tha  grid  of  tha  form 


j-1«s  -v2 


V2  >  ,  1  <  iti  <  3 


approach**  uniform  (Labaaqua)  maaaura  on  tha  cuba  (0#1)a  in  Kolmogorov  matric  at  tha  rata 
j-1  -  Under  Assumption  3,  F  look*  like  uniform  meaaure  on  sufficiently  snail 

cubes,  eo  the  rate 

d  -n-1/d 
n 

is  at  least  attainable.  However,  if  one  chooses  tha  knots  randomly  (i.e.  if  t1 ,  t2,  ... 
are  independent  and  identically  distributed  with  the  distribution  F) ,  then  the  rate  can  be 
improved  to 

d  -  0D(n‘1/2  )  , 

n  p 

where  0p  means  "big  oh  in  probability,”  See  equation  (2.4.3)  in  Gaenssler  and  Stute  [6]. 

He  conjecture  that  d  »  n  1  is  obtainable  in  any  dimension,  but  the  construction  of  knot 
n 

sequences  attaining  thia  rate  appears  to  be  a  nontrivial  problem,  in  general,  if 

d  «  n  r 
n 


for 


r  >  0,  and  if  aleo 


X  -  , 

(so  that  the  optimal  rate  is  obtained),  then  Assumption  2  requires  that 

q  >  d(5-2r)/(4r)  . 

This  is  a  stronger  assumption  than  the  minimal  one  required  for  Theorem  5.1  (i.e.  that 
q  >  d)  unless  r  >  S/6. 
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<■  Extension  to  Larger  Bggjjjg,  Suppose  that  F  and  0  satisfy  Assumptions  3  and  4,  and 
that  Q*  is  a  domain  satisfying 

n  c  a*  , 

e.g.  O'  -  ■!* .  Consider  tha  smoothing  splins  estimate  g^  obtained  by  minimising  over 

h  e  *f^(Q' )  the  quadratic  form 

Z  I  (a.  -  h(t.  ))2  +  X  l  J  (D*W>)2dt  . 
k  x  *  |a|-m  O' 

The  only  difference  between  g^  end  gR^  is  that  the  objective  function  involves 
)  norms  of  mth  order  derivatives  rather  than  L2<Q)  norms.  If 
O'  -  **,  then  g^  is  called  a  "thin  plate"  smoothing  spline  [22],  or  Laplacian 
smoothing  spline  [241.  Our  convergence  rates  will  be  in  terms  of  tha  expected  value  of  the 
square  of  the  L2 (F^)  seminorm  of  the  error,  vis. 

Kx  -  *  t  / 

-  *  [  n"1  J  (,^(1^)  -  ,  (tk>)2]  , 

JC“I 

with  Tni  defined  by  replacing  g^  with  g^. 

We  define  an  n  x  n  matrix  by 

<6-11  Kxi- 

where  i  r"  ♦  W^(Q* )  is  the  smoothing  spline  operator  for  the  domain  O’  (see 

Definition  1.1).  It  is  easily  checked  that 

TnX  *  /  ^n*<»‘V»  -  ***«  +  n-’o^UA;,)2]  . 

See,  for  example,  equation  (1.7)  of  craven  and  ffahba  [5].  A  simple  argument  (Lemma  4.1  of 
[5])  shows  that  if  gC  W^(Q'),  then 

JlS'.(g(A„))  -  g]  2dF  4  X  /  (g<B,(u))2du 
“An  n  fll 

Wj(O') 

for  all  n  >  1  and  all  X  >  0.  Hence,  to  obtain  an  upper  bound  for  T'  , 

n  a 
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2 

it  la  only  necessary  to  bound  Tr(A^) .  Ma  shall  do  this  by  showing  that  tha  aiganvaiusa 

of  ara  boundad  by  tha  corresponding  aiganvaluas  of  A^y  tha  matrix  ohtainad  by 

dalating  tha  primas  in  (6.1 ) .  a  simpla  variant  of  tha  proof  of  Issmu  5.S  than  givas  an 
2 

uppar  bound  on  Tr  assuming  that  f  and  Q  satisfy  tha  aaaumption*. 

Dafina  tha  quadratic  form 

B  (UfVj  -  /  uvdf 
n  n 

for  aithar  u,v  «  w*{0)  or  u,v  e  w“(fl* )  . 

Ma  assuma  that 


d  C  AC  ft*  . 
n  ■*  — 

Also  dafina  tha  quadratic  forms 

A  ,(u,v)  -  B  (u,v)  +  X(u,v)  , 

nX  n  ^(0) 

A*  -  (U,V)  -  B  (U,V)  +  X(U,V)  _  , 

B  l£(0*) 

with  domains  M^(ft)  and  M^<0' ) ,  raspactivaly.  It  follows  from  the  proof  of  Proposition 
2.1  that  Aay  and  A’ny  ara  strictly  positive  definite.  Furthermore,  the  codimension  of 
the  null  space  of  Bn  is  n  for  both  domains,  and  tha  Rayleigh  quotients 
»ntu,u)/An^(u,u)  and  Bn(u,u)/V^(u,u)  ara  boundad  by  1.  Hence,  there  ara  n  positive 

eigenvalues. 


(6.2) 

for  tha  Rayleigh  quotient 


“nXI  *  UnX2  >  —  ”  UnXn  >  0 


Bn<u,u)/An^(u,u).  Bare,  tha  eigenvalues  are  defined 


recursively  by 


*  sup  (Bn(u,u)  «  u  6  An^(u,u)  -  1>  , 

where  tha  suprsmum  is  obtained  at  u^,  and  assuming  (un^,  un^)  has  been  defined  for 
1  «  j  <  k  -  1, 


nXk 


sup  (Bn<u,u)  i  u  B  An^(u,u)  »1,  and 

An^(u,un^)  «  0  for  1  <  j  <  k  -  1}  . 


Bee  Theorem  1.2,3  of  Weinberger  (21).  Bimilary,  we  let 


(6.3) 


Km  *  Ku  *  —  ”  ^Xn  >  0 
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b«  the  eigenvalues  for  the  Rayleigh  quotient  u, u)/A^(u,u) .  For  a  more  complete 
account  of  these  eigensysteas  and  further  applications  (in  one  dimension)  see  Speckman 
(14),  Section  5.  The  most  important  fact  for  our  purposes  is  that  (6.2)  (respectively 
(6.3))  are  the  eigenvalues  for  the  matrix  A^a  (respectively  A^) .  To  see  this,  note 
that  the  variational  equation  of  Proposition  2.1  may  be  written  as 

\a(v'*»x>  ’  Vv't>  Vvew2(n> 

where  C  e  w”(fl)  is  any  function  satisfying 

_£  (An>  “  Z  . 

Hence,  if  for  some  Y  >  0  we  have 

AnX  *  “  ^ 

then  we  may  take  C  “  Y  ^g  ^  so  that 

AnX(v'9„X)  *  ▼“1»nX‘v.«nX>,  V  v  e  M"tQ>  ' 

from  which  it  follows  that  Y  is  an  eigenvalue  of  the  Rayleigh  quotient  B  /A  .  . 

n  n  a 


Laws  6.1.  Let  m  >  d/2  and  suppose  that  0  C  S*.  Then  the  eigenvalues  in  (6.2)  and 
(6.3)  satisfy 

K*  <  “nXk  f0r  1  4  *  4  n  • 

Proof.  The  assumption  m  >  d/2  implies  that  ( 0)  and  1^(0*)  may  be  embedded  in 
C  (0)  and  C  (O’),  respectively  (Theorem  4.6.1(e)  of  Triebel  117)).  Hence, 

B  ,  A  .,  and  A1  .  are  well  defined,  and  so  is  the  restriction  operator 

I)  nA  (In 

R  :  w“(fl')  *  w£<0),  i.e.  f Ru) (t)  -  u( t)  for  all  t  «  0,  all  u  e  W*(0’). 

Then  the  following  inequalities  are  obvious: 


B 

B  (Ru,Ru)  >  (- 
"  A 

An^(RU,Ru)  > 


(u,u) 


;x<u'ul 

B  (u,u) 
n 

a;a(u,u) 


)  AnA(RU,Ru) 


a;x(u'u) 


for  all  u  e  w"(fl’)/{ 0).  The  lemma  now  follows  from  the  "Mapping  Principle,"  Theorem 

3.6.1  of  Weinberger  (23).  • 
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Theorem  6.2.  Suppose  Assumptions  1/2,3/  and  4  hold  (for  the  domain  8),  and  that 

m  >  3d/2.  Then,  for  any  domain  8*  2  there  exists  a  finite  constant  C  >  0 

depending  on  m,  8,  8',  and  F,  and  an  Integer  nQ  such  that  for  all  n  >  nQ  and  all 

Xe  IX  ,A^]  , 

n  n 


4  x><« 


2  -1  -d/2m 
♦  c  o  n  A 


"“(O') 


Proof.  In  view  of  Lemma  6.1  and  the  remarks  preceding  it,  it  suffices  to  show  that 

Tr  A*  -  0(  A'd/2m)  . 

n  a 

uniformly  in  A  e  [X  ,A  )  as  n  ♦  •*.  Since 
n  n 


n_1  o2  Tr  x2x  -  E  [n_1  l  <Snle>2(tj)] 


j-1 

.2 


nXI' 


B,S„xfV(Fn)  ' 

i  n 


and  by  Assumption  4  and  Lemma  5.5, 


uniformly  in  X«  (An,Anl '  At  suffices  to  prove  that 
(6.4) 


*  )  -  'V.'lm  1  -  . 


2  n . -  “2' 

d, 

>2, 


uniformly  in  A  e  [X  ,A  ].  To  this  end,  apply  Lemma  4.2  (i)  and  Lemma  5.5  to  obtain 
n  n 


|E/(Snxf)  d[F-Fn)|  <  Vn  E.SnXe.d 


-d  n-’x-3^2" 
n 

-  <d  X-<V“Hn'1X-<1/2m)  . 
n 


Equation  (6.4)  now  follws  from  Assumption  2. 


* 
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